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ABSTRACT 



The extent to which the instruction, assessment, and 
Maryland School Performance Assessment Program (MSPAP) test preparation 
activities of language arts teachers aligned with the Maryland Learning 
Outcomes (MLOs) and the reading and writing tasks on the MSPAP was studied in 
the 1996-97 school year. The reading and writing activities of 280 elementary 
and middle school language arts teachers were collected in the fall and 
spring and analyzed with a coding scheme. In general, the majority of the 
language teachers' classroom activities reflected some of the MLOs. However, 
the extent to which the classroom instruction and assessment activities 
reflect the many characteristics of the MSPAP tasks is more limited. Slightly 
more than one-third of the instruction and assessment tasks were classified 
at one of the two highest score levels on the MSPAP scale, and 49% of the 
reading and 66% of the writing MSPAP test preparation activities were 
classified at one of the two highest levels. These results support the 
results from language arts questionnaires suggesting that teachers have made 
changes in their instruction based on the MLOs and the MSPAP. (Contains 18 
tables.) (SLD) 
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Impact of the Maryland School Performance Assessment Program (MSPAP): 
Evidence from Language Arts Classroom Activities 



Purpose 

The purpose of this paper is to determine the extent to which the language arts teachers’ classroom 
instruction, assessment, and MSPAP test preparation activities collected during the 1996-97 school year 
are aligned with the Maryland Learning Outcomes (MLO’s) and the reading and writing tasks on 
MSPAP. Each of the teachers’ reading and writing activities were analyzed using a coding scheme 
designed to provide information about the extent to which the teachers’ activities reflect the MLO’s, the 
response type required of the student, the level of integration with other subject areas, the amount of 
group work, and the overall similarity to MSPAP tasks. In addition, analyses were conducted to 
determine if there are any differences between the teachers’ instruction activities and assessment activities 
in terms of their alignment with the MLO’s, and to determine if any differences exist in the teachers’ 
activities across grade levels. 



Methodology 



Data Collection and Instruments 

Language arts teachers were asked to collect samples of instruction and assessment activities used in 
their classroom during the 1996-97 school year The activities were collected at two time points. In 
December of 1996, teachers were asked to send in approximately 5 language arts instruction activities, 5 
language arts assessment activities, and I sample of a language arts scoring scheme used in their 
classrooms from September to December 1996 Similarly, in the spring they were asked to send in 
another set of 5 instruction activities, 5 assessment activities, and 1 scoring scheme used from January to 
June 1997. In addition, teachers were also asked to send a sample of a MSPAP test preparation activity 
used prior to the administration of MSPAP. If a teacher taught more than one language arts class, they 
were requested to obtain these materials from a typical class that they taught. 

A data collection form was developed to obtain information from each teacher regarding the 
classroom from which the activities were selected. The data collection forms asked teachers to indicate 
the grade level, the nature of the students’ ability levels for the language arts class (e.g., heterogeneous 
ability group, homogeneous ability group, exclusively special education, exclusively gifted and talented), 
and the nature of the content taught in the class (e.g., language arts, reading only, writing only, English) 
Each teacher completed a form and returned it along with their activities for each collection period. 
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Teachers were also provided with a set of labels to attach to their classroom activities. On these 
labels teachers were instructed to indicate whether the activity was used for instructional purposes, 
assessment purposes, or scoring/evaluating purposes. They also were asked to indicate the source of the 
activity (e.g., teacher developed, commercial resource or textbook, county-developed, state-developed). 

Procedures for Coding Language Arts Classroom Activities 

The classroom instruction, assessment, and MSPAP test preparation activities were analyzed for each 
language arts teacher using a coding scheme designed to provide information about the extent to which 
the activities reflect the Maryland Learning Outcomes for reading and writing, the overall similarity to 
MSPAP-like tasks, and a variety of other features (e.g., response type required of students, integration 
with other subject areas, etc.). The Maryland Learning Outcomes and the format and content of MSPAP 
served as the basis for the coding schemes that were developed for the analysis of the classroom activities. 
An additional coding scheme was developed to analyze the scoring schemes. Features of this coding 
scheme included the type of scoring scheme (e.g., letter/numerical grades, checklists, point systems, and 
scale levels), the components of the activity that are being evaluated, and the evaluator of the activity 
(e.g., teacher, student, peer). However, only the results for the classroom activities (instruction, 
assessment, and MSPAP test preparation activities) are included in this paper. 

A total of four raters coded the classroom activities. A formal training session was conducted to 
familiarize the raters with the coding scheme using a sample set of pre-coded activities. The raters coded 
another set of sample activities independently and their codes were compared and discussed by the group. 
After the formal training was complete, pairs of raters individually coded sets of classroom activities from 
a school for a given collection period (fall or spring). To ensure that all raters shared a common 
understanding of the coding scheme, subsets of raters met to discuss their discrepancies and reached a 
consensus on the codes for each activity within the set. This consensus coding was done for a sample of 
elementary schools and a sample of middle schools. Thus, for a small percentage of classroom activities 
(19%) one set of codes, agreed upon by two or more raters, was obtained. 

After it was determined that the raters reached a shared understanding of the coding scheme and were 
proficient in applying it to a variety of classroom activities, each rater individually coded sets of 
classroom activities. Approximately 25% of the sets of classroom activities (an elementary or middle 
school teacher’s activities from either fall or spring) were coded individually by two raters. An overall 
adjusted rate of agreement between the raters was then calculated. 1 The adjusted rate of agreement was 

1 Percent agreement was considered to be too lenient of an index of rater agreement because for several of the 
categories to be coded there were a range of options that could be selected. As an example, for the response type 
component, one to twenty-two response types could be selected for an activity. However, the majority of the 
activities had between one and six response types selected. A simple percent agreement based on each of the 
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85% across all instruction, assessment, and MSPAP test preparation reading activities and 88% across all 
instruction, assessment, and MSPAP test preparation writing activities. 

Sample 

A subset of schools in the entire sample (approximately 57%) were asked to participate in the 
collection of the classroom activities. Overall, 51 schools with a total of 372 language arts teachers were 
asked to participate in this aspect of the study. Some or all of the teachers from 44 of the schools 
participated, resulting in a school participation rate for classroom activities of 86%. This represents 
schools from 15 different systems/counties in Maryland. Within these 44 schools, 280 language arts 
teachers sent in all or a subset of the activities requested (75%). 

Description of Language Arts Classes and Teachers 

As described previously, language arts teachers were asked to send in a total of approximately 10 
instruction activities, 10 assessment activities, 2 scoring schemes, and 1 MSPAP test preparation activity 
used in their classrooms throughout the 1996-97 school year. If a teacher taught more than one language 
arts class, they were requested to select activities from a typical class they taught. A total of 280 language 
arts teachers sent in a sample of their classroom activities. The following sections provide a description 
of the type of language arts classroom from which activities were obtained, the heterogeneity of language 
arts classes, information on sample sizes by grade and type of activity, and the sources of the activities. 

Type of Language Arts Class 

On the data collection form for each collection period, teachers were asked to indicate the type of 
language arts class from which their sample of classroom activities was selected. As indicated in Table 1, 
the majority of the elementary classes were labeled as “language arts” classes (81%), while a slightly 
lower percentage (67%) of the middle school classes were “language arts” classes. Other types of classes 
in the middle school were labeled by the teachers as English (14%), reading (13%), or writing (3%), The 
percentages of elementary classes of these types were lower (1%, 8%, and 2%, respectively). 



twenty-two response types would have inflated the index for rater agreement. Thus, an adjusted percent agreement 
was used. Three sources of information were taken into consideration to obtain the adjustment. For example, the 
frequency distributions for the number of total response types coded per activity, the percentage of times each 
category was coded, and raters’ knowledge of the coding scheme and nature of the activities. 
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Table 1. 

Type of Language Arts Classes from which Classroom Activities were Selected 





All Grades 
n=316* 


Elementary 

n=203 


Middle 
n=l 13 


Language Arts 


76% 


81% 


67% 


Reading 


10% 


8% 


13% 


Writing 


2% 


2% 


3% 


English 


5% 


1% 


14% 


Not Indicated 


6% 


8% 


3% 



* Note: This number is larger than the 280 teachers who sent in classroom 
activities because 36 teachers had a change in the type of language arts class 
taught from fall to spring. 



Heterogeneity of Language Arts Class 

Also on the data collection form, teachers were asked to indicate the heterogeneity of the students in 
the language arts class from which their sample of classroom activities were selected. As indicated in 
Table 2, 78% of the elementary classes compared to only 36% of the middle school classes were 
classified by the teacher as heterogeneous. A larger percentage of middle school classes (43%) than 
elementary classes (12%) were classified as homogeneous, either on-grade, above-grade, or below-grade 
level. In addition, a larger percentage of middle school classes (15%) compared to elementary classes 
(6%) were classified as either exclusively special education or gifted and talented. 



Table 2. 

Heterogeneity of Language Arts Classes from which Classroom Activities were Selected 





All Grades 
n=320* 


Elementary 

n=199 


Middle 

n=121 


Heterogeneous 


62% 


78% 


36% 


Homogeneous 


23% 


12% 


43% 


On-Grade 


11% 


6% 


20% 


Above-Grade 


8% 


3% 


17% 


Below-Grade 


4% 


3% 


6% 


Exclusively Special Education 


8% 


5% 


12% 


Exclusively Gifted & Talented 


2% 


1% 


3% 


Not Indicated 


6% 


5% 


7% 



* Note: This number is larger than the 280 teachers who sent in classroom activities because 
40 teachers had a change in the heterogeneity of the language arts class taught from fall to spring. 

Teachers and Classroom Activities by Grade Level 

On average across the entire 1996-97 school year, approximately 13 classroom activities were 
collected per teacher. For each grade level, Table 3 indicates the number and percentage of teachers at 
each grade level who sent in classroom activities and also the total number and percentage of all 
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classroom activities received. For example, 39 2 nd grade teachers sent in a total of 464 classroom 
activities, which represents 14% of the teachers and 13% of the classroom activities. The percentages 
across grades for the number of teachers and the number of activities are somewhat similar, although a 
slightly smaller percentage of off-grade teachers (2, 4, and 7) versus on-grade teachers (3, 5, and 8) sent 
in activities. This difference can most likely be explained because teachers who taught both on- and off- 
grades were asked to send in materials from their on-grade class. 



Table 3. 

Teachers and Classroom Activities bv Grade Level 



Grade 


Teachers 


Activities 




Number 

(n=283)* 


Percentage 


Number 

(n=3559) 


Percentage 


2 


39 


14% 


464 


13% 


3 


54 


19% 


745 


21% 


4 


39 


14% 


489 


14% 


5 


47 


17% 


613 


17% 


7 


43 


15% 


530 


15% 


8 


61 


22% 


718 


20% 



* Note: This number is larger than the 280 teachers who sent in classroom activities 



because 3 teachers had a change in grade taught from fall to spring. 



Type of Classroom Activity 

Teachers were provided with labels to attach to each classroom activity indicating the type of activity 
(i.e., instruction, assessment, MSPAP test preparation). In the first two columns of Table 4, the number 
and percentage of activities for each type is shown. Across all grades (2, 3, 4, 5, 7, and 8) there was a 
total of 1900 instruction activities and 1208 assessment activities. For grades 3, 5, and 8 (on-grades only) 
there was a total of 1 13 MSPAP test preparation activities. The table also includes a category called “not 
coded”. These 338 activities were not coded for one of two reasons. One reason for not coding an 
activity was because it pertained strictly to another content area such as social studies or science. The 
other types of tasks that were not coded include puzzles, games, word searches etc., or teacher notes 
and/or lesson plans that had no clear indication as to what the students were required to do. 

The last three columns of Table 4 indicate the number and percentage of teachers providing each type 
of material as well as the average number of each type of activity per teacher. As an example, about 53% 
(1900 out of 3559) of the activities received were labeled as instruction, and nearly all the teachers (98%) 
(273 out of 280) teachers sent in at least one instruction activity. Thus, the mean number of instructional 
activities provided per teacher was approximately 7. It should be noted that the percentages among the 
grade levels for each of the types of activities were similar. 
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Table 4. 

Type of Classroom Activity 





Activities 

(n=3559) 


Teachers 

(n=280) 


Mean 
Number of 
Activities 




Number 


Percentage 


Number 


Percentage 


Per Teacher 


Instruction 


1900 


53% 


273 


98% 


6.96 


Assessment 


1208 


34% 


233 


83% 


5.18 


MSPAP Test Preparation (3,5,8) 


113 


3% 


57 


35%* 


1.98 


Not Coded 


338 


9% 


134 


48% 


2.52 



♦Note: This percentage is based only on the on-grade teachers (3, 5, 8) and not on the full sample of 280 teachers. 



Sources of Classroom Activities 

Teachers were also asked to indicate the source of each activity on the label. Table 5 indicates the 
sources of the instruction, assessment, and MSPAP test preparation activities across all grades. A 
comparison of results for each grade indicated quite similar results. On average, 40% of the instruction 
activities were teacher-developed, and a little over one-third (36%) were from textbook or commercial 
resources. Nearly half of the assessment activities (47%) were teacher-developed and 32% were from 
textbook or commercial resources. Approximately equal percentages for instruction and assessment 
activities were county developed (5% and 6% respectively). The percentages of instruction and 
assessment activities obtained from state-level materials, such as MSPAP Release Tasks, Maryland 
Consortium Tasks, and Maryland Performance-Based Exemplars, were very small. 

When examining the MSPAP test preparation activities, the sources were somewhat different than for 
the instruction and assessment activities. The percentage of teacher-developed MSPAP test preparation 
activities and textbook or commercial resources for MSPAP test preparation activities was the same 
(23%), and the percentage of county-developed MSPAP test preparation activities was only slightly lower 
(19%). Therefore, as compared to the instruction and assessment activities, fewer MSPAP test 
preparation activities were teacher-developed or obtained from textbooks or commercial resources, 
however more test preparation activities were county-developed. In addition, there were more MSPAP 
Release Tasks (6%), and other state-level materials (12%) used for MSPAP test preparation compared to 
instruction and assessment activities. 
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Table 5. 

Sources of Classroom Activities 





Instruction 

n=1900 


Assessment 

n=1208 


MSPAP 
Test Prep 
n=l 13 

(3, 5, 8 only) 


Teacher/Other Teacher/ School Developed 


40% 


47% 


23% 


Textbook/Commercial Resources 


36% 


32% 


23% 


County/ Another County Developed 


5% 


6% 


19% 


Teacher and Textbook/ Teacher and County Developed 


10% 


7% 


11% 


MSPAP Release Tasks 


<1% 


<1% 


6% 


MD Consortium/Exemplars 


1% 


1% 


12% 


Other/Teacher and Student/Class (Scoring) 


<1% 


<1% 


0% 


Cannot Be Determined 


7% 


8% 


7% 



Results 



Overall Description of Reading and Writing Activities 

The coding scheme for analyzing the language arts classroom activities consisted of two components: 
a reading and a writing component. Depending on what was required of the students, the language arts 
instruction, assessment, and MSPAP test preparation activities were coded for reading only, for writing 
only, or for both reading and writing. A reading activity was considered to be an activity that required 
students to either read a paragraph or more from a textbook or other resource, or to respond to various 
reading skill exercises such as vocabulary exercises and defining literary terms. A writing activity 
involved writing at least a paragraph or responding to writing skill exercises including spelling, grammar, 
capitalization, punctuation, and/or alphabetizing. 

Table 6 shows the percentage of language arts classroom activities that were considered to be 
“reading only”, “writing only”, “reading and writing linked”, and “reading and writing not linked”. 
Overall, there was a larger percentage of “reading only” activities compared to “writing only” activities 
(40% versus 25%), and about one-third (33%) of the language arts activities consisted of a reading 
component connected to a writing component, that is, “reading and writing linked”. In general, an 
activity coded as “reading and writing linked” required students to read a passage and respond to at least 
one question based on the reading by writing one or more paragraphs. Only a very small percentage of 
classroom activities (1%) consisted of both a reading component and a writing component but were not 
connected, that is, “reading and writing not linked”. 
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Table 6. 

Percentages of Reading and Writing Activities 



(n=3221) 


Reading Only 


40% 


Writing Only 


25% 


Reading & Writing Linked 


33% 


Reading & Writing Not Linked 


1% 



Overall, 74% of all the language arts instruction, assessment, and MSPAP test preparation activities 
included at least a reading component, and 59% of all the language arts activities included at least a 
writing component. 

A comparison of the percentages of reading and writing instruction, assessment, and MSPAP test 
preparation activities indicated a higher percentage of MSPAP test preparation activities that were 
“reading and writing linked” activities (45%) compared to the instruction and assessment activities (32% 
and 34%, respectively). By grade level for the instruction activities, 5 th and 8 th grade had slightly higher 
percentages of activities that were “reading and writing linked” (38%) compared to other grade levels 
such as 2 nd grade (26%) and 7 th grade (24%). For the assessment activities and the MSPAP test 
preparation activities, differences among grades levels were more noticeable. For example, 45% of the 8 th 
grade assessment activities were “reading and writing linked” compared to only 29% for 7 th grade; and 
67% of the 8 th grade MSPAP test preparation activities were “reading and writing linked” compared to 
only 39% for 3 rd grade and 26% for 5 th grade. Differences between on- and off-grade levels were larger at 
the middle school levels than at the elementary school levels. 

Reading and Writing Skill Exercises and Tasks 

A further distinction in reading and writing activities was made with respect to the purpose of the 
activity. In general, some activities have a purpose as defined by the Maryland Learning Outcomes 
(MLO’s); these activities will be referred to as “tasks”. Other activities that do not contain a purpose, as 
defined by the MLO’s, will be referred to as “skill exercises”. There are a few exceptions to this 
classification scheme, and they are noted within the following descriptions of “tasks” and “skills”. 

To be coded as a reading “task”, the activity needed to reflect at least one “Purpose of Reading” 
and/or at least one “Stance of Reading” as defined by the MLO’s. The Purposes of Reading are: Reading 
for literary experience, reading to be informed, and reading to perform a task. The Stances of Reading 
are: Global understanding, interpretive stance, personal stance, and critical stance. (The reading MLO’s 
will be described in more detail in a later section of this paper.) In addition, a reading “task” must also 
contain at a least a paragraph of reading material. The only exception to this classification were activities 
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that did not have a reading purpose as defined by the MLO’s but did reflect at least one reading stance, 
typically global understanding. They required students to read one or more paragraphs (not obtained from 
novels, plays, nonfiction books, articles, etc.) for basic reading comprehension. Therefore, these activities 
were labeled as “reading comprehension” in this coding scheme and were classified as reading “tasks”. 

A reading “skill exercise” does not have an MLO reading purpose nor an MLO reading stance. 
Examples of reading skill exercises included vocabulary exercises, defining literary terms, and circling 
the adjectives in a reading passage. It should be noted that reading activities may have questions related 
to “skill exercises” as well as reading “tasks”. In these cases, the reading activity was coded as both a 
“skill exercise” and a “task”. 

To be coded as a writing “task”, the activity required the student to write at least one paragraph. In 
many cases, the activity explicitly stated that the length of writing should be at least a paragraph. 
However, when it did not, other pieces of information were used to determine if the length of writing 
would most likely be a paragraph or more. For instance, when the student was asked to write a letter, 
poem, story, speech, article, journal entry, etc., then it was assumed that the response would be at least a 
paragraph. On the other hand, when the student was asked a series of short-answer questions that require 
a phrase or sentence after reading, it was not considered a writing “task”. In addition to the length of the 
writing activity as a criteria for writing tasks, a writing “task” typically reflected at least one of the MLO 
Purposes of Writing: Writing to inform, writing to persuade, or writing to express personal ideas. 
However, a small number of writing “tasks” did not reflect one of these MLO’s and were labeled as 
“summarizing/interpreting reading material”. In these cases, the writing activity included a reading 
“task” and required students to summarize, interpret, and/or critically analyze the reading passage. 
Although these writing “tasks” did not reflect any of the MLO-defined writing purposes, they required 
students to reflect on what they read by summarizing or interpreting the material. 

A writing “skill exercise” did not reflect any of the MLO Purposes of Writing nor 
“summarizing/interpreting reading material”. Instead, writing “skill exercises” included spelling, 
grammar, capitalization, punctuation, and/or alphabetizing. It should be noted that writing activities may 
have consisted of both “skill exercises” as well as one or more writing “tasks”. In these cases, the writing 
activity was coded as both a “skill exercise” and a “task”. 

Table 7 indicates the percentages of activities that were coded as. “skill only” and “task only” and 
“both skill and task”. The results are similar for both the reading and writing activities. Only a small 
percentage of the activities were coded as “skill only” (12% and 18%, respectively), and approximately 
75% of all reading and writing activities were coded as “task only”. When considering the activities that 
were “both skill and tasks”, the percentage of activities with at least a “task” increases to 87% for all 
reading activities and 82% for all writing activities. 
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Table 7. 

Percentages of Reading and Writing Skill Exercises and Tasks 





Reading Activities 


Writing Activities 




n=2402 


n=1923 


Skill Only 


12% 


18% 


Task Only 


73% 


78% 


Both Skill & Task 


14% 


4% 



The percentages of reading and writing “skill exercises” and “tasks” are somewhat similar for 
instruction, assessment, and MSPAP test preparation activities, with the MSPAP test preparation 
activities having slightly larger percentages of “task only” activities (85% for reading and 94% for 
writing) compared to instruction and assessment activities (approximately 72% for reading and 75% for 
writing). Variations between grade levels are small. 

Reading and Writing Components 

All reading and writing activities were coded in terms of several components. This section describes 
each of the components and provides summary information related to the categories within each of the 
components as well as some significance tests to examine differences between grade levels and type of 
activity (i.e., instruction versus assessment) The components are: reflection of the Maryland Learning 
Outcomes, type of response required of the student, group work, integration with other subject areas, and 
similarity to MSPAP tasks. 

Maryland Learn in e Outcomes 

Reading Activities . All instruction, assessment, and MSPAP test preparation reading activities were 
coded in terms of whether they included the Purposes for Reading and the Stances of Reading as defined 
by the MLO’s. With regard to the reading purposes, the MLO’s indicate that “students will demonstrate 
their ability to vary their orientation to the text by interacting with a variety of texts for different 
purposes”. The three reading purposes are: I ) Reading for literary experience, which involves reading 
novels, plays, poems, or short stories; 2) reading to be informed, which involves reading content texts, 
articles, and editorials; and 3) reading to perform a task, which involves following directions and requires 
some action of the student. 

Table 8 presents the percentages for each reading purpose. Out of the activities coded as reading 
“tasks”, 91% reflected at least one of the three MLO reading purposes. Reading for literary experience 
was the most frequent purpose for reading, in that 70% of the reading tasks reflected this purpose, while 
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reading to perform a task was the least frequent purpose (3%). Notice that the percentages for the three 
MLO reading purposes sum to more than 91%. This is because some of the reading “tasks” (about 12%) 
were coded for more than one purpose. For instance, reading historical fiction was coded as “reading for 
literary experience” and also coded as “reading to be informed”. The remaining 9% of the reading 
“tasks” that did not have an MLO-defined reading purpose were coded to have a purpose labeled “reading 
comprehension”, which included tasks that involved basic comprehension of at least a paragraph. As 
mentioned previously, these reading tasks did not have an MLO-defined reading purpose, however they 
did reflect at least one MLO reading stance, which was typically global understanding. All reading 
"tasks" were additionally coded as to the type of reading material students were required to read. About 
62% of all reading "tasks" involved reading novels or short stories, and about 14% involved a reading 
passage of one or more paragraphs that was not from a novel, short story, play, poem, non-fiction book or 
newspaper article. Other reading "tasks" required that students read plays, poems, or songs (7%); articles 
or newspapers (7%), and non-fiction books such as biographies (5%). 

The second column in Table 8 shows the percentages of all reading activities, including reading “skill 
exercises” and “tasks”, that reflected each purpose of reading. These percentages indicate the extent to 
which all reading activities collected from the teachers reflected an MLO Purpose of Reading and/or 
Stance of Reading. In other words, 80% of all reading activities contained at least one MLO purpose and 
an additional 8% contained the “reading comprehension” purpose, therefore the remaining 12% of the 
reading activities were “skill exercises" onl> Recall that reading skill exercises do not have an MLO 
reading purpose nor the purpose of reading comprehension. Of this 12%, over half (53%) were coded as 
“contextual vocabulary” (defining vocabulan words that are set in a reading passage), 21% were coded as 
“vocabulary not in a passage”, 14% were coded as “literature skills” (e.g., defining or identifying literary 
terms such as irony, theme, or metaphor). 11% were coded as “reference skills” (e.g., using the card 
catalog, or internet), and 16% were coded as “other skills” such as phonics, pronunciation, reading aloud 
for practice, and circling adjectives in a reading passage. A small number of reading “skill exercises” 
were coded in more than one of these categories. 
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Table 8. 

Maryland Learning Outcomes Reflected in Reading Activities 





Reading Tasks 
(n=2102) 


All Reading Activities 
(n=2402) 


Reading Purposes 


At Least One MLO Purpose 


91% 


80% 


For Literary Experience 


70% 


62% 


To Be Informed 


30% 


26% 


To Perform a Task 


3% 


3% 


Other Purpose (not an MLO) 


Reading Comprehension 


9% 


8% 


Reading Stances 


At Least One MLO Stance 


100% 


88% 


Global and/or Interpretive 


95% 


84% 


Personal 


32% 


28% 


Critical 


20% 


17% 



Reading activities were also coded according to whether they reflected the MLO reading stances. 
The four MLO stances of reading allow students to demonstrate their ability to interact with a variety of 
texts. The stances include global understanding (ability to construct an initial, global understanding of the 
text by considering theme, character’s or story’s main problem, author’s overall purpose or point of view, 
etc.); developing interpretation (clarify, verify, and revise one’s understanding by considering such 
aspects as plot and character development, by organizing text information, or by following directions to 
complete a task); personal reflection/response (consider and compare the author’s point of view with 
student’s own point of view, consider and compare new information from the text with the student’s own 
background knowledge); and critical stance (identify and analyze the author’s perspective and craft in 
order to form and substantiate a critical response). 

As indicated in Table 8, all reading “tasks” (100%) had at least one MLO reading stance. This is 
because the inclusion of a stance is required of a reading “task” as defined previously. Once again, each 
reading task could consist of multiple reading stances, and approximately 37% of the reading tasks did 
include more than one reading stance. Nearly all tasks (95%) explicitly required a global and/or 
interpretive stance 2 , almost one-third (32%) required a personal stance, and one-fifth (20%) required a 
critical stance. Further, all reading “tasks” were coded as to whether the task explicitly required students 
to provide examples, evidence, or support based on their reading. Approximately 27% of the reading 



2 The “global understanding” reading stance and the “developing interpretation” reading stance were combined for 
the purposes of this analysis. The reading material was not usually included with the activity, and it was often 
difficult to determine whether the questions on the reading activity reflected a global stance, an interpretive stance, 
or both. Thus, the two categories were combined. 
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tasks did require this type of explicit support. Similar to the results for the purposes of reading, a column 
of percentages is also included in Table 8 that represents the percentage of all reading activities that 
required a reading stance. Overall, 88% of all reading activities required at least one MLO reading 
stance. 

Writing Activities . All instruction, assessment, and MSPAP test preparation writing activities were 
coded in terms of whether they included the Purposes for Writing and the Stages of the Writing Process as 
defined by the MLO’s. With regard to the writing purposes, the MLO’s indicate that “students will 
demonstrate ability to write for various audiences and to address a variety of purposes”. The three writing 
purposes are: 1) Writing to inform (convey information using factual or personal data), 2) writing to 
persuade (convince or refute by supporting a point of view), and 3) writing to express personal ideas 
(communicate feelings and imagination). 

Table 9 presents the percentages for each writing purpose. Out of the activities classified as writing 
“tasks”, 74% reflected at least one of the three MLO-defined writing purposes. Writing to inform was the 
most frequent purpose for writing (41%), 22% of the “tasks” reflected writing to express personal ideas, 
and the least frequent purpose was writing to persuade (12%). The “choice” category (3%) consisted of 
those tasks for which students had a choice in what they were to write about, thus allowing for a choice in 
the purpose of writing. Only a very small percentage of the writing “tasks” had more than one MLO 
writing purpose coded. The remaining 26% of the writing “tasks” that did not have an MLO-defined 
purpose for writing were coded to have a purpose defined as “summarizing/interpreting reading material”. 
As previously described, these writing “tasks” were not classified as having an MLO-defined writing 
purpose because the purpose of the questions in the task did not adequately match any of the three MLO 
writing purposes. However, they did require students to summarize, interpret, critically analyze or reflect 
on the reading passage. 

The second column in Table 9 indicates the percentages across all writing activities, including all 
writing “skill exercises” and “tasks”, that reflected each purpose of writing. These percentages indicate 
the extent to which all writing activities collected from the teachers reflected an MLO purpose of writing. 
In other words, 60% of all writing activities contained at least one MLO purpose and an additional 21% 
contained the “summarizing/interpreting reading material” purpose, therefore the remaining 19% of the 
writing activities were “skill exercises” only. Recall that writing “skill exercises” do not have an MLO 
reading purpose nor the purpose of “summarizing/interpreting reading material”. Of this 19% of reading 
skill exercises, approximately 76% included “writing mechanics exercises, not in paragraph form” (e.g., 
series of sentences/phrases for which students are required to identify incorrect verb tenses, spelling 
errors, or alphabetizing and punctuation errors), 20% included “writing mechanics exercises in paragraph 
form” (e.g., students are required to correct capitalization errors, provide punctuation, complete a 
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salutation in a letter, etc.), 5% involved writing hyperboles, similes, and metaphors, and 6% involved 
writing or rewriting phrases or sentences for stylistic purposes. A small number of writing “skill 
exercises” were coded into more than one of these categories. 



Table 9. 

Maryland Learning Outcomes Reflected in Writing Activities 





Writing 

Tasks 

(n=1569) 


Writing 

Activities 

(n=1923) 


Writing Purposes 


At Least One MLO Purpose 


74% 


60% 


To Inform 


41% 


33% 


To Persuade 


12% 


10% 


To Express Personal Ideas 


22% 


18% 


Choice 


3% 


2% 


Other Purpose (Not an MLO) 


Summarizing/Interpreting Reading Material 


26% 


21% 


Writing Process 


All 4 Stages (Prewrite, Draft, Revision, Proofread) 


20% 


16% 


2 or 3 Stages 


Prewrite, Draft 


40% 


32% 


Draft, Revision, Proofread 


1% 


1% 


Draft, Proofread 


1% 


1% 


1 Stage (Draft Only) 


36% 


29% 


Cannot Be Determined 


3% 


2% 



The writing activities were also coded in terms of the writing process. As defined by the MLO’s, 
there are four stages of the writing process. They are: prewriting (generating ideas for topics, setting 
purposes, ordering ideas, or identifying audiences), drafting (writing a first draft for a purpose and an 
audience), revising (using self, peer, and teacher input to revise the draft; considering completeness and 
appropriateness of style), and proofreading (considering correctness and language in use). To gain more 
information about the prewriting stage of the writing process, the specific type of prewriting component 
was coded for each writing "task". The pre writing component found most frequently was the 
development or use of graphic organizers (32%). Graphic organizers included outlines, character maps, 
story webs, and other graphic devices that helped students organize their information prior to writing. 
Other prewriting components included answering questions or prompts (23%), individual brainstorming 
(18%), activities in other content areas such as creating a table or graph (15%), and group brainstorming 
or class discussion (7%). 

In order for a writing activity to be coded for each of the stages, the activity had to explicitly mention 
the stage of the writing process. Therefore, the percentages in Table 9 for writing process may be 
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underestimates. It is possible that teachers required their students to proceed through all, or at least more 
than one, of the writing stages; however, the teacher may not have made an explicit indication of this on 
the actual activity, and thus it would not be coded as such. The only exception to this is when the teacher 
indicated that “the writing process was used” without indicating each of the four stages explicitly. For 
these instances, the activities were coded for all four stages of the writing process. However, this 
occurred for only a small portion of the tasks (3%). 

As indicated in Table 9, 20% of the writing tasks were coded as requiring all four stages of the 
writing process, i.e., prewriting, drafting, revising, and proofreading. An additional 42% of the writing 
activities involved two or three stages of the writing process. Therefore, a total of 62% of the writing 
tasks involved at least two stages of the writing process, that is going beyond writing a draft only. The 
“cannot be determined” category included tasks for which only student work was included, thus it was 
difficult to determine whether the work represented students’ first draft or students’ final draft after 
proceeding through each of the writing process stages. The results shown in the second column indicate 
that 16% of all writing activities included all four stages of the writing process, and an additional 34% 
included two or three of the stages. 

Comparisons bv grade level and type of activity . To determine if there are differences in percentages 
between grade levels and differences in percentages for instruction activities versus assessment activities, 
a repeated measures analysis of variance with one between factor (grade) and one within factor (type of 
activity) was conducted. Table 10 below shows an average proportion at each grade level of reading 
activities per teacher that reflected at least one MLO reading purpose and at least one MLO reading 
stance. For this analysis, only those teachers who sent in both instruction and assessment activities were 
included, which represents 77% of all language arts teachers who sent in classroom materials (215 out of 
280) 3 . After transforming the data using an arcsin transformation, the results of the analysis for the MLO 
reading purposes indicate no significant difference between grades (F(5, 209)=2.028, p=.076) and no 
significant difference within teachers for instruction versus assessment activities (F(l, 209)=0.773, 
p=.380). 

Similarly, for the analysis of the MLO reading stance, there is no significant difference within 
teachers for instruction versus assessment activities (F(l, 209)=2.463, p=. 1 1 8). However, there is a 
significant difference at the .05 level between grade levels (F(5, 209)=2.265, p=.049). The interaction 
between grade level and type of material was not significant (F(5, 209)=1.302, p=.264). Tukey HSD 
post-hoc analyses were conducted to determine which differences between grades were significant. The 



3 MSPAP test preparation activities were not included in this analysis because of the small sample size. A 
comparison among the three types of activities (instruction, assessment, and MSPAP test preparation) would have 
reduced the sample to only 38 teachers, which represents only 14% of all language arts teachers. 
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significant differences occurred within the assessment activities for two pairs of grade levels. A larger 
percentage of assessment activities for both 4 th grade (off-grade) and 8 th grade (on-grade) teachers 
reflected at least one MLO reading stance compared to 7 th grade (off-grade) teachers (p=.034 and .047, 
respectively). As shown in Table 10, only 66% of the 7 th grade (off-grade) teachers’ assessment activities 
reflected an MLO reading stance versus 88% of the 4 th grade (off-grade) teachers’ activities and 87% of 
the 8 th grade teachers’ activities. It should be noted that although a difference between grade levels was 

identified as significant, the proportion of variability attributable to grade level is very small (R^=.051) 



Table 10. 

Maryland Learning Outcomes for Reading Activities bv Grade and Type of Activity 





All 

grades 

(n=215) 


2 nd 

grade 

(n=29) 


3 rd 

grade 

(n=42) 


4*^ 

grade 

(n=28) 


5^ 

grade 

(0=34) 


grade 

(n=33) 


8 th 

grade 

(n=49) 


At least one MLO reading purpose 


Instruction activities 


.80 


.81 


.78 


.84 


.81 


.73 


.83 


Assessment activities 


.76 


.75 


.75 


.79 


.79 


.62 


.84 


At least one MLO reading stance 


Instruction activities 


.88 


.87 


.87 


.95 


.84 


.86 


.88 


Assessment activities 


.83 


.87 


.86 


.88 


.84 


.66 


.87 



Table 11 below presents an average proportion at each grade of writing activities per teacher that 
reflected at least one MLO writing purpose and at least two stages of the writing process. Once again, 
only those teachers who sent in both instruction and assessment activities were included in the analysis. 
This represents 70% of all language arts teachers (197 out of 280). After transforming the data using an 
arcsin transformation, the results of the analysis for the MLO writing purpose indicate a significant 
difference between grades (F(5, 191)=2.753, p=.020) and also a significant difference within teachers for 
instruction versus assessment activities (F(l, 191)=6.742, p=.010). There was no significant interaction 
between grade level and type of activity (p=.629). Therefore, across all grades more instruction activities 
reflected at least one MLO writing purpose compared to assessment. As can be seen in Table 11, an 
average of 64% of the teachers’ instruction activities reflected a writing purpose whereas only 51% of the 
teachers’ assessment activities did. Tukey HSD post-hoc analyses indicated no significant pairwise 
differences between grade levels, however it appears that for the assessment activities a larger percentage 
of 4 th and 5 th grade teachers’ activities have a writing purpose compared to the other grade levels. Once 
again, while difference between grades and within instruction and assessment activities for teachers were 
significant, practically the differences were small (R^=.034 for within type of activity and R^=.067 for 
between grades). The analysis of the writing process indicates no significant difference within teachers 
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for instruction versus assessment activities (F(l, 191)=.262, p=.609). There is also no significant 
difference between grade levels (F(5, 191)=.914, p=.473). 



Table 11. 

Maryland Learning Outcomes for Writing Activities bv Grade and Type of Activity 





All 


2 nd 


¥ 


4 th 


5 th 


7“ 


8 «h 




grades 

(n=197) 


grade 

(n=28) 


grade 

(n=39) 


grade 

(n=28) 


grade 

— (n=32) 


grade 

(n=28) 


grade 

(n=42) 


At least one MLO writing purpose 
Instruction activities 


.64 


.59 


.61 


.69 


.69 


.62 


.63 


Assessment activities 


.51 


.45 


.45 


.69 


.64 


.42 


.44 


At least two stages of writing 
















process 

Instruction activities 


.50 


.43 


.50 


.59 


.49 


.51 


.48 


Assessment activities 


.45 


.45 


.41 


.56 


.51 


.36 


.44 



Response Required of Student 

The MSPAP tasks that are scored for reading and writing require students to respond in a variety of 
ways including providing words, phrases, several sentences or a paragraph or more, and developing 
graphic organizers. Therefore, the classroom reading and writing activities from the language arts 
teachers were coded according to how the students were required to respond to the activity. 

Reading Activities . Overall, 17% of the reading activities required only traditional response types 
(e.g., selected response formats and providing a word or phrase), 22% required only a limited written 
response (e.g., response less than a paragraph), and 59% required at least one extended response (e.g., one 
or more paragraphs, graphic organizer). The remaining 2% required other types of responses (e.g., 
drawing pictures and artwork, numerical response). The percentage of times each response type was 
coded is provided in Table 12. Each reading activity could potentially include one or more response types. 
Not surprisingly, a larger percentage of reading "tasks" included more than one response type (59%) as 
compared to the percentage of reading "skill exercises" (29%). The first column of the table represents all 
reading activities, and the second and third columns represent reading "skill exercises" and reading 
"tasks", respectively. 
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Table 12. 

Response Type for Reading Activities 





All Reading 
Activities 
(n=2402) 


Reading 

Skill 

(n=637) 


Reading 

Task 

(n=2102) 


Selected Response 


25% 


57% 


17% 


Word/Phrase 


29% 


42% 


24% 


Response Less than Paragraph 


43% 


18% 


45% 


One or more Paragraphs 


42% 


3% 


47% 


Graphic Organizer 


32% 


8% 


35% 


Drawing Pictures & Artwork 


43% 


2% 


12% 


Oral/Class/Group Discussion 


10% 


3% 


10% 


Other 


9% 


2% 


10% 


Cannot Determine 


2% 


1% 


1% 



As indicated in the table, across all reading activities only a small percentage had a selected response 
format (25%) and required students to provide a word or phrase (29%). Almost half of the reading 
activities involved a response less than a paragraph (but more than a word or phrase) (43%), a response of 
one or more paragraphs (42%), or drawing pictures and/or artwork (43%). Approximately one-third of 
the activities required students to develop a graphic organizer. As can be expected, there were differences 
in the types of responses for a reading "skill exercise" versus a reading "task". For example, 57% of the 
"skill exercises" compared to only 17% of the "tasks" had a selected response format, whereas only 3% of 
the "skill exercises" compared to 47% of the "tasks" required a written response of one or more 
paragraphs. 

A few differences occurred when comparing instruction, assessment, and MSPAP test preparation 
activities. Assessment activities contained more selected response formats (38%) than instruction 
activities (17%). Also, graphic organizers were used more often with instruction activities (35%) than 
assessment activities (26%). A larger percentage of MSPAP test preparation activities required students 
to write a paragraph or more (58%) or to provide a response of less than a paragraph (61%) as compared 
to instruction (41% and 43%, respectively) and assessment activities (42% and 40%, respectively). 
Moreover, a larger percentage of graphic organizers (41%) were used with MSPAP test preparation 
activities as compared to assessment activities (26%). 

With respect to differences between grade levels, the elementary and middle grades differed in the 
frequency of selected response formats and responses that required a word or phrase. For assessment 
activities, 7th and 8th grades had higher percentages of selected response formats (about 50%) compared 
to 2nd, 3rd, 4th, and 5th grades (about 32%). For instruction activities, 7th and 8th grades had higher 
percentages of activities that required a word or phrase (about 42%) than elementary grades (about 27%). 
In addition, assessment activities that required one or more paragraphs occurred more frequently in the 
4th, 5th, and 8th grades (about 50%) compared to the 2nd, 3rd, and 7th grades (about 35%). For the 
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MSPAP test preparation activities, a larger percentage of activities that required one or more paragraphs 
occurred in the 8th grade (69%) compared to the 3rd grade (55%) and 5th grade (42%). 

One other difference between grades is in the use of graphic organizers. In general, elementary grade 
levels had more activities that consisted of graphic organizers than the middle grades, although the 
specific differences vary quite a bit for each type of activity. For instance, about 40% of the instruction 
activities at the elementary grades included graphic organizers compared to only 31% at 7th grade and 
27% at 8th grade. For assessment activities, there are also differences between on- and off-grade levels. 
For the 2nd and 4th grades, more than 40% of the assessment activities include graphic organizers, at the 
3rd and 5th grades the percentages decrease to 27%, and at the middle grades the percentages drop to 
10% (7th grade) and 14% (8th grade). Finally, for MSPAP test preparation activities, the percentage of 
graphic organizers is largest at the 3rd grade (64%) and smallest at the 5th and 8th grades (38% and 31%, 
respectively). 

Writing Activities . Overall, 18% of the writing activities required only traditional response types 
(e.g., selected response format, making corrections, providing a word or phrase, providing a sentence or 
two), 40% of the writing activities required only limited written responses (e.g., one paragraph, several 
connected paragraphs), and 41% of the writing activities required at least one extended written response 
(e.g., essay, story, play, letter, speech, editorial, article). A different set of response types were coded for 
writing "skill exercises" and writing "tasks", because as discussed in an earlier section, one criteria for a 
writing activity to be coded as a writing "task" was that students were required to write at least one 
paragraph. About 37% of the "skill exercises" had more than one response type, and about 13% of the 
writing "tasks" were coded for more than response type. The first column in Table 13 provides the 
percentages of all writing activities for each response type. Only 12% of all writing activities had a 
selected response format, whereas about one-third (32%) involved writing one paragraph. In addition, 
15% of the writing activities required two or more connected paragraphs and 16% required that students 
write a letter. 

An examination of the "skill exercises" indicates that over half (54%) required students to select a 
response, and approximately one-third were coded as making corrections, providing a word or phrase, and 
providing a sentence or more (but less than a paragraph). For the writing "tasks", the most frequent 
response types are: one paragraph (39%), letters (20%), and connected paragraphs (18%). 
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Table 13. 

Response Type for Writing Activities 





All Writing Activities 
(n=1923) 


Writing Skills 
(n=427) 


Writing Tasks 
(n=1569) 


Selected Response 


12% 


54% 


— 


Make Correction 


7% 


32% 


— 


Word or Phrase 


7% 


33% 


— 


Sentence or More 


7% 


30% 


— 


Other Content (Numerical 
Value, Graph , Table) 


<1% 


1% 


— 


One Paragraph 


32% 


— 


39% 


Connected Paragraphs 


15% 


— 


18% 


Essay 


7% 


— 


8% 


Story 


7% 


— 


9% 


Letter 


16% 


— 


20% 


Play / Poem 


6% 


— 


7% 


Editorial / Critique / 
News Article 


3% 


— 


4% 


Speech 


2% 


— 


3% 


Other 


5% 


— 


6% 


Choice 


3% 


— 


4% 



When comparing the instruction and assessment activities for "skill exercises", a larger percentage of 
assessment activities (63%) compared to instruction activities (45%) required students to select a 
response. Within the instruction activities, there are a few differences between grades. A smaller 
percentage of 8th grade activities required students to select a response (27%) compared to 7th grade 
(47%), whereas a larger percentage of 8th grade activities required students to make corrections (50%) 
compared to 7th grade (29%). With regard to assessment activities, the middle grades have smaller 
percentages of selected response types (about 47%) in comparison to the elementary grades (ranging from 
60% in 5th grade to 76% in 2nd grade) 

For the writing "tasks", the percentages of response types for instruction, assessment, and MSPAP 
test preparation activities are quite similar For instance, students were asked to respond in the form of 
one paragraph for 41% of the instruction activities, 37% of the assessment activities, and 34% of the 
MSPAP test preparation activities. The one noticeable difference is in the response type of "letters". 
Students were more frequently asked to write letters for MSPAP test preparation activities (41%) 
compared to instruction activities (17%) and assessment activities (22%). This is probably due to the 
frequency of letter-writing on MSPAP. 

Grade-level differences for writing "tasks" are noticeable between the upper grades and lower grades. 
In general, as the grades increased there were fewer required responses of one paragraph and more 
required responses of several connected paragraphs. These differences occurred for the both the 
instruction and assessment activities. For the MSPAP test preparation activities, a larger percentage of 
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editorials, critiques, or news articles (13%) and speeches (10%) were required at the 8th grade, whereas 
0% of these response types occurred at the 3rd grade and only 4% occurred at the 5th grade. However, a 
larger percentages of "letters" were required at the 3rd grade (46%) and 5th grade (52%) compared to 8th 
grade (29%). 

Group Work 

The MLO's indicate that providing students with opportunities to work collaboratively with others is 
an important aspect of the learning environment. Therefore, the reading and writing classroom activities 
from the language arts teachers were coded as to whether they involved individual, pair, and/or group 
work. Across all reading activities, 14% explicitly required students to work in pairs or groups. In almost 
two-thirds of these instances, the activity involved a combination of some pair or group work and some 
independent activity. Similarly, across all writing activities, 16% explicitly required students to work in 
pairs or groups, and in over 80% of these instances, the activities involved a combination of pair work, 
group work, and/or independent activity. It is possible that the percentage of actual pair or group work 
for the reading and writing activities may be actually higher than these percentages since pair or group 
work was only coded it if was explicitly indicated on the activity that students were to work together. 

Integration with Other Subject Areas 1 

The majority of the actual MSPAP tasks that assess reading and writing are integrated with other 
subject areas including science, social studies, and mathematics. Therefore, the classroom reading and 
writing activities were analyzed in terms of whether they were integrated with other subject areas. For 
example, a reading activity may have asked students to read an article about a new discovery in science. 
The reading activity may have science questions, but also reading-related questions that require taking a 
critical stance, such as "what did the author have to know in order to write this article?" An example of a 
writing activity integrated with social studies may be one in which students are asked to write a 
persuasive letter about their views on school prayer. 

Table 14 provides the results for the integration of reading and writing activities with social studies, 
science, math, and art/music/social skills. Notice that the integration between reading and writing is not 
presented in this table. Information related to the integration between reading and writing activities was 
presented in Table 6, that is, 34% of all the language arts teachers' activities integrated reading and 
writing. Overall as shown in Table 14, the amount of integration with other subject areas is quite similar 
for reading and writing. About 33% of the reading activities and 32% of the writing activities were 
integrated with at least one other subject area. For those reading and writing activities that were 
integrated with other subject areas, social studies was the most common subject area (25% for reading 
and 22% for writing). Science was the second most commonly integrated subject (9% for reading and 
10% for writing). 
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Integration of Reading and Writing with Other Subject Areas 





Reading 

Activities 

(n=2402) 


Writing 

Activities 

(n=1569) 


Integration with At Least 
One Subject Area 


33% 


32% 


Social Studies 


25% 


22% 


Science 


9% 


10% 


Math 


2% 


3% 


Art / Music / Social Skills 


3% 


3% 


Choice 


1% 


2% 


Cannot Determine 


<1% 


<1% 



A comparison across the types of activities for reading shows that MSPAP test preparation activities 
had the largest percentage of activities integrated with at least one other subject area (61%) compared to 
35% for instruction activities and only 27% for assessment activities. The two subject areas with the 
highest percentages for MSPAP test preparation were social studies (44%) and science (25%). Across 
grade levels, the percentages varied somewhat. For instruction, smaller percentages of activities 
integrating at least one subject area occurred at the 2 nd and 3 rd grades (23% and 27%, respectively) 
compared to approximately 40% in 4 th , 5 th , 7 th , and 8 th grades. For assessment, the percentages of 
activities integrated with at least one subject area varied from elementary to middle grades and from on- 
and off-grades, with a low of 18% at the 7 th grade and a high of 34% at the 8 th grade. For MSPAP test 
preparation, 8 th grade had the largest percentage of activities that were integrated with at least one subject 
area (67%), while 64% of the 3 rd grade activities and 50% of the 5 th grade activities were integrated. In 
general, across grade levels, when a reading activity did consist of some integration with other subjects it 
was most frequently with social studies, and in some cases science for MSPAP test preparation activities. 

The results for the writing activities are somewhat similar to the reading activities. Across all grades, 
the percentages of integration for writing instruction and assessment activities (32% and 27%) were lower 
than for the MSPAP test preparation activities (71%). With respect to differences between grades for 
instruction, the lower elementary grades (2 nd and 3 rd ) have lower percentages of activities that are 
integrated with at least one subject area (21% and 22%, respectively) compared to the upper elementary 
and middle grades (ranging from 33% to 42%). Differences between grades occur for elementary versus 
middle and on- versus off-grades. For instruction and assessment activities, the most frequently 
integrated subject is social studies, and science is the next most frequently integrated subject. The 
remaining subject areas are rarely integrated with writing. This is consistent across grade levels. 
However, for the MSPAP test preparation activities in the 3 rd and 5 th grades, in addition to social studies 
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and science being integrated with writing, mathematics is also frequently integrated with writing (23% for 
3 rd grade and 28% for 5 th grade). 

Similarity to MSPAP Tasks 

The reading and writing activities were coded with respect to their similarity to MSPAP tasks. Two 
five-level scales were developed that considered the MLO’s for reading and writing and also the type of 
response students were required to provide. The levels of the “MSPAP scale” for reading and the 
“MSPAP scale” for writing were similar. For instance, the first level of the scale, labeled as “not like 
MSPAP- skill”, included activities that involved only “skill exercises” with no “tasks”. The second 
through the fifth levels of the MSPAP scale were used when the activities were classified as “tasks”, and 
they ranged from “not like MSPAP- task” to “MSPAP high”. Figure 1 below provides descriptors of a 
typical task for each of the five levels of the reading “MSPAP scale”. 
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Figure 1. 

Description of the Five Levels of the Reading “MSPAP Scale" 



“MSPAP scale” Levels 
for Reading 


Descriptors of Levels for Reading 


Not like MSPAP-skill 


Reading activities with only reading “skill exercises” 

Does not reflect any MLO reading purposes 

Does not reflect any MLO reading stances 

Includes vocabulary exercises, reference skills, pronunciation, etc. 


Not like MSPAP-task 


Reading activities with a reading “task” 

Does not reflect any MLO reading purposes 

Reflects only the global understanding MLO reading stance 

Involves reading one or more paragraphs 

Requires answering selected response or short answer items 

Requires very few inferences 


MSPAP low 


Reading activities with a reading “task” 

Reflects at least one MLO reading purpose 
Reflects at least one MLO reading stance, typically global 
understanding 

Involves reading one or more paragraphs 

Requires answering in the form of one or more sentences 

Primarily requires low-level inferences 


MSPAP medium 


Reading activities with a reading “task” 

Reflects at least one MLO reading purpose 
Reflects at least one MLO reading stance, with at least one reading 
stance beyond global understanding (interpretive, critical, or 
personal stance) 

Typically involves answering about 2 or 3 questions that require a 
response of a few sentences, a paragraph, or more 
Requires some high level inferences 

Contains all the elements of a MSPAP task, but is not as extended 


MSPAP high 


Reading activities with a reading “task” 

Reflects at least one MLO reading purpose 
Reflects at least one MLO reading stance, with at least one reading 
stance beyond global understanding (interpretive, critical, or 
personal stance) 

Typically involves answering more than three questions that require 
a response of a paragraph or more 
Requires high level inferences 

Requires examples, details, and/or reasons to support the response 
Contains all the elements of a MSPAP task and is extended 
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Table 15 provides the percentages of reading activities at each of the five levels. As shown, 25% of 
the reading activities are considered to be “not like MSPAP”, whether they are “skill exercises” only or 
reading “tasks”, whereas 75% of the reading activities have at least some elements of MSPAP. The 
majority of the activities are at a “MSPAP low” level (42%), while 23% are at a medium and 10% are at a 
high level. 



Table 15. 

Reading MSPAP Scale 



All Reading Activities 
(n=2402) 


Not like MSPAP-skill 


12% 


Not like MSPAP-task 


13% 


MSPAP Low 


42% 


MSPAP Medium 


23% 


MSPAP High 


10% 



The writing “MSPAP scale” also has five levels with the first level reflecting only writing “skill 
exercises” and the remaining four levels reflecting writing “tasks” ranging from “not like MSPAP- task” 
to a “MSPAP high”. Figure 2 provides descriptors of typical tasks at each of the five levels of the writing 
“MSPAP scale”. 
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Figure 2. 

Description of the Five Levels of the Writing “MSPAP Scale” 



“MSPAP scale” Levels 
for Writing 


Descriptors of Levels for Writing 


Not like MSPAP-skill 


Writing activities with only writing “skill exercises” 

Does not reflect any MLO writing purposes 
Does not reflect any stages of the MLO writing process 
Includes exercises involving spelling, grammar, capitalization, 
punctuation, etc. 


Not like MSPAP-task 


Writing activities with a writing “task” 

Does not reflect any MLO writing purposes 
Does not reflect any stages of the MLO writing process, other than 
writing a draft 

Example Task: writing a draft that summarizes and/or interprets 
reading material 


MSPAP low 


Writing activities with a writing “task” 

Either reflects at least one MLO writing purpose, or reflects some 
or all stages of the writing process 

Example Task: writing a draft of a letter to a congressman stating a 
position on a particular issue 

Example Task: developing a graphic organizer, writing a draft that 
discusses how characters in the story dealt with conflict, and 
revising the draft 


MSPAP medium 


Writing activities with a writing “task” 

Reflects at least one MLO writing purpose, and reflects some stages 
of the writing process 

Example Task: organize thoughts on an issue using a graphic 
organizer and write a letter to a congressman stating a position on 
a particular issue 


MSPAP high 


Writing activities with a writing “task” 

Reflects at least one MLO writing purpose, and reflects all stages of 
the writing process 

Example Task: organize thoughts on an issue using a graphic 
organizer, write a letter to a congressman stating a position on a 
particular issue, then revise and proofread the letter 



Table 16 provides the percentages of writing activities at each of the five levels. As shown, the first 
two levels of the scale combined represent 29% of the activities that are considered to be “not like 
MSPAP”, either “skill exercises” only or writing “tasks”. Therefore, the remaining 70% of the writing 
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activities have at least some elements of MSPAP. There are 3 1 % of the writing activities at the third 
level, which includes tasks that either have an MLO-defined writing purpose or reflect at least some of the 
stages of the writing process. The fourth level includes 26% of the activities which reflect both an MLO- 
defined writing purpose and two or three stages of the writing process. Finally, the fifth level, which 
contains 14% of the activities, represents activities that reflect an MLO-defined writing purpose and all 
four stages of the writing process. 



Table 16. 

Writing MSPAP Scale 



All Writing Activities 
n=1923 



Not like MSPAP-skill 1 8% 

Not like MSPAP-task: no MLO, draft only 11% 

MSPAP low: MLO or some/all stages of writing process 3 1 % 

MSPAP medium: MLO Mid some stages of writing process 25% 

MSPAP high: MLO and all stages of writing process 14% 



Comparisons bv grade level and type of activity . To determine if there are differences in the level of 
similarity to MSPAP tasks between grade levels and within instruction and assessment activities, a 
repeated measures analysis of variance with one between factor (grade) and one within factor (type of 
activity) was conducted. Table 17 below shows an average MSPAP level for teachers at each grade level. 
For this analysis, only those teachers who sent in both instruction and assessment reading activities were 
included, which represents 77% of all language arts teachers who sent in classroom materials (215 out of 
280). Similar to the previous analysis for the reading MLO’s, the MSPAP test preparation activities were 
not included in this analysis because of the small sample size. The results of the analysis for the reading 
“MSPAP scale” indicate no significant difference within teachers for instruction versus assessment 
activities (F(l, 209)=. 187, p=.666). However, a significant difference occurred between grade levels 
(F(5, 209)=4.920, p<.001). The interaction between grade and type of activity was not significant 
(p=.326). 



Table 17. 

Similarity to Actual MSPAP Reading Tasks bv Grade and Type of Activity 





All 


2 nd 


3 rd 


4 * 




-^zr- 


8® 




grades 

(n=215) 


grade 

(n=29) 


grade 

(n=42) 


grade 

(n=28) 


grade 

(n=34) 


grade 

(n=33) 


grade 

(n=49) 


Instruction activities 


3.04 


2.80 


2.88 


3.32 


3.04 


2.86 


3.26 


Assessment activities 


3.01 


2.74 


3.01 


3.44 


3.08 


2.55 


3.17 
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Tukey HSD post-hoc analyses indicated two significant pairwise differences between grades for 
instruction reading activities. The average level on the reading “MSPAP scale” was higher for both the 
4 th grade and 8 th grade compared to the 2 nd grade (p=.044 and p=.041, respectively). Three significant 
pairwise differences occurred between grade levels for the assessment reading activities. Both the 4 th 
grade and the 8 th grade had higher average levels on the reading “MSPAP scale” than the 7 th grade 
(p=.002 and p=.034, respectively). In addition, the 4 th grade also had a higher average MSPAP level 
compared to 2 nd grade (p=.047). However, the difference between grades was small in a practical sense 

(R 2 =.105). 

A repeated measures analysis of variance was also conducted to examine differences in the writing 
activities between grades and within type of activity. Table 18 indicates average writing MSPAP levels 
for teachers at each grade level and type of activity. Once again, only those teachers who sent in both 
instruction and assessment activities were included in this analysis, which represents 70% of all language 
arts teachers (197 out of 280). The results of the analysis indicate a significant difference within teachers 
for instruction versus assessment activities (F(l, 191)=6.294, p=.013). Instruction activities have a higher 

MSPAP level than assessment activities, although practically the difference is very small (R 2 =,017). 
There is also a significant difference between grade levels (F(5, 191)=3.622, p=.004). Tukey HSD post- 
hoc analyses indicate one significant pairwise difference between grades. For assessment activities, there 
was a higher MSPAP level at the 4 th grade level compared to the 3 rd grade level (p=.018). However, the 

amount of total variance accounted for by the grade level is small (R 2 =.087). There was no interaction 
between grade and type of activity (p=.641). 



Table 18. 

Similarity to Actual MSPAP Writing Tasks bv Grade and Type of Activity 





All 


p 


F 


— 


5“ 


yth 


8 th 




grades 

(n=197) 


grade 

(n=28) 


grade 

(n=39) 


grade 

(n=28) 


grade 

(n=32) 


grade 

(n=28) 


grade 

(n=42) 


Instruction activities 


3.09 


2.82 


3.00 


3.37 


3.18 


3.01 


3.15 


Assessment activities 


2.87 


2.68 


2.58 


3.36 


3.10 


2.60 


2.92 
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Discussion 



Overall Comparisons Between Reading and Writing 

In general, the majority of the language arts teachers' classroom activities in this analysis reflected 



some of the Maryland Learning Outcomes. Approximately 80% of all reading activities required students 
to read for literary experience, read to be informed, and/or read to perform a task. In addition, 88% of all 
reading activities also required students to analyze the reading passage in a global, interpretive, personal, 
and/or critical stance, with the majority of these activities taking a global and/or interpretive stance (84%) 
and only 35% taking a personal and/or critical stance. For the writing activities, 60% required students to 
write to inform, write to persuade, and/or write to express personal ideas, and an additional 21% required 
students to summarize and interpret reading material, although the latter is not an MLO writing purpose. 
Approximately 50% of all writing activities in this analysis required students to use at least two stages of 
the writing processes, that is, going beyond writing a draft to include a prewriting activity, revision, 
and/or proofreading. 

The extent to which the Maryland Learning Outcomes and the characteristics of MSPAP are reflected 
in the reading and writing activities is shown by the results of the coding for the “MSPAP scales”. 
Overall, the percentages of reading activities at each level of the reading “MSPAP scale” compared to the 
writing activities at each level of the writing “MSPAP scale” are quite similar. When combining the first 
two scale levels, “not like MSPAP-skills*' and “not like MSPAP-task”, about 25% of reading activities 
and 29% of writing activities are considered to be “not like MSPAP”. The remaining reading and writing 
activities have at least some elements of MSPAP tasks. There is a slightly larger percentage of reading 
activities that are classified as “MSPAP low“ (42%) compared to writing activities (31%) and there are 
slightly smaller percentages of reading activities classified as “MSPAP medium” and “MSPAP high” 
(23% and 10%), compared to writing activities (25% and 14%). Thus, although the majority of the 
classroom activities reflected the Mary land Learning Outcomes to some extent, only 33% of the reading 
activities and 39% of the writing activities received one of the two highest levels of the MSPAP scale. 

For other reading and writing components such as integration with other subject areas and group 
work, the percentages were quite similar for both content areas. Only a small percentage of reading and 
writing activities included an explicit mention of either pair or group work (about 15%). Recall that this 
percentage may be an underestimate of the extent of group work used in teachers’ classrooms, since pair 
or group work was only coded if there was an explicit mention on the activity sent in by the teacher. The 
percentage of tasks that were integrated with at least one subject area was about 33% for both reading and 
writing. The most frequently integrated subject area for reading and writing was social studies, and the 
next most frequent was science. 
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Comparisons Within Type of Activity for Reading and Writing 

In general, the instruction and assessment activities were similar with regard to the MLO’s and the 
MSPAP scale for both reading and writing. When differences between instruction and assessment 
activities were detected in the analysis, the differences were small. As might be expected, there were 
some relatively large differences for the MSPAP test preparation activities compared to the instruction 
and assessment activities. As an example, 30% of the reading instruction activities and 36% of the reading 
assessment activities received one of the two highest “MSPAP scale” levels, whereas 49% of the reading 
MSPAP test preparation activities received one of the two highest levels. In addition, 42% of the writing 
instruction activities and 35% of the writing assessment activities received one of the two highest 
“MSPAP scale” levels, whereas 66% of the writing MSPAP test preparation activities received one of the 
two highest levels. As another example, approximately 70% of the MSPAP test preparation reading and 
writing activities were integrated with at least one other subject area, whereas only about 35% of the 
instruction and assessment activities were integrated. 

Comparison Among Grade Levels, and Between Classroom Activities and Teacher Questionnaire 
Responses 

In general, there were few differences among the grades for the instruction and assessment activities 
and when differences were obtained they were small. This is consistent with the results obtained from the 
teacher questionnaire (Lane, Ventrice, Cerrillo, Parke & Stone, 1999). As an example, 4 th and 8 th grade 
reading assessment tasks reflected the reading stances more often than 7 th grade assessment tasks, and 4 th 
and 8 th grade reading assessments were more aligned to the MSPAP tasks than were the 7 th grade 
assessment tasks. These findings are somewhat consistent with the questionnaire results in that all 
elementary grades and 8 th grade teachers indicated that their instruction and assessment activities were 
more similar to the reading learning outcomes than did the 7 th grade teachers. 

The average level on the “MSPAP scale” for reading and writing activities was a “MSPAP low”. 
Even though approximately 75% of the classroom activities reflected at least some elements of an actual 
MSPAP task, only about 35% were classified at the “MSPAP medium” or “MSPAP high” level. 
Teachers’ responses on the language arts questionnaire with regard to current reading and writing 
instruction indicated a somewhat greater degree of alignment between their classroom activities and the 
Maryland Learning Outcomes. On a 4-point Likert scale, where a “1” indicated no alignment and a “4” 
indicated a great amount of alignment, in general, teachers’ responses averaged at about a “3” level. 
Comparison Between Language Arts and Mathematics Classroom Activities 

Overall, when examining the similarity between the classroom activities and MSPAP tasks, the 
reading and writing activities appear to be slightly more aligned to MSPAP than the mathematics 
activities. As an example, 11% of the reading instruction activities and 16% of the writing instruction 




30 

07/16/99 




K30 T2M 



32 



activities were classified as “not like MSPAP- skill” and an additional 11% and 8%, respectively, were 
classified as “not like MSPAP- task”. In contrast, 39% of the mathematics activities were classified as 
“not like MSPAP-computation/equation” and an additional 6% were classified as “not like MSPAP- 
traditional word problems” (Lane, Parke, & Stone, 1998, 1999). 

Conclusion 

The analysis of the classroom activities suggests that the majority of the classroom instruction and 
assessment activities provided by the language arts teachers reflect the Maryland Learning Outcomes. 
However, the extent to which the classroom instruction and assessment activities reflect the many 
characteristics of the MSPAP tasks is more limited. Slightly more than one-third of the classroom 
instruction and assessment tasks were classified at one of the two highest score levels on the MSPAP 
scale. However, 49% of the reading and 66% of the writing MSPAP test preparation activities were 
classified at one of the two highest score levels on the MSPAP scale. These results help support the 
results from the language arts questionnaire suggesting that teachers have made changes in their 
instruction based on the Maryland Learning Outcomes and MSPAP (Lane, Ventrice, Cerrillo, Parke & 
Stone, 1999). However, their classroom activities may not reflect the MLO’s and MSPAP to the extent 
to which they had indicated on the questionnaire. 
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